Generative Adversarial Networks Research Articles

In-Domain GAN Inversion for Faithful Reconstruction and Editability.

Generative Adversarial Networks (GANs) have significantly advanced image synthesis through mapping randomly sampled latent codes to high-fidelity synthesized images. However, applying well-trained GANs to real image editing remains challenging. A common solution is to find an approximate latent code that can adequately recover the input image to edit, which is also known as GAN inversion. To invert a GAN model, prior works typically focus on reconstructing the target image at the pixel level, yet few studies are conducted on whether the inverted result can well support manipulation at the semantic level. This work fills in this gap by proposing in-domain GAN inversion, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model. In this way, we manage to sufficiently reuse the knowledge learned by GANs for image reconstruction, facilitating a wide range of editing applications without any retraining. We further make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property. Such a trade-off sheds light on how a GAN model represents an image with various semantics encoded in the learned latent distribution.

Multi-stage generative adversarial networks for generating pavement crack images

The application of machine learning techniques in pavement health monitoring based on computer vision has greatly improved the accuracy and efficiency in the detection of pavement distress levels and categories. However, a persistent challenge in this field is the issue of sample imbalance, primarily arising from the scarcity of cracked pavement images, which hampers their effectiveness in road maintenance engineering. To address this issue and enhance the fast and stable generation of high-quality crack images for engineering purposes, this study proposes two frameworks based on Generative Adversarial Networks (GAN): Multi-Stage GAN-v1 and Multi-Stage GAN-v2. These frameworks break down the complex task of directly generating high-quality images into a series of incremental steps, gradually increasing the image resolution from initially generated low-precision images. Both versions, v1 and v2, consist of multiple sequentially connected generation units, with each unit utilizing the Wasserstein Generative Adversarial Network-Gradient Penalty (WGAN-GP). Furthermore, v2 has the additional capability of generating pavement crack images of specified types and simultaneously providing crack segmentation labels. This feature significantly enhances the practical applicability of the generated data in engineering contexts. In a comprehensive case study, the evaluation results clearly illustrate the superior image generation quality from the two proposed frameworks. Moreover, the results from ablation experiments, involving the training of nine state-of-the-art crack semantic segmentation and object detection networks using both generated images and real images, demonstrate the effective utility of these generated images for training pavement distress detection networks.

Implementing generative adversarial networks for increasing performance of transmission fault classification

<p>An electrical power system is a network that facilitates the sourcing, transfer, and distribution of electrical energy. In the traditional power system, there are eleven types of faults that can occur in the system. This paper focuses on the classification of these faults over a stretch of 100 kilometres. The dataset used is synthetic and generated from a simulated model using MATLAB/Simulink software. Data augmentation is carried out during training to improve the accuracy of the classification. An indirect training approach through generative adversarial network (GAN) is used to classify these overhead transmission line faults. The random forest (RF) classification is used as the base learning model on the original dataset and it achieves accuracy of 84%. However, the base learner RF when used on GAN model generated augmented faulty data, it performs exceptionally well achieving 99% accuracy. One of the recent state-of-art methods is compared with this approach.</p>

Detection of individual brain tau deposition in Alzheimer's disease based on latent feature-enhanced generative adversarial network

The conventional methods for interpreting tau PET imaging in Alzheimer's disease (AD), including visual assessment and semi-quantitative analysis of fixed hallmark regions, are insensitive to detect individual small lesions because of the spatiotemporal neuropathology's heterogeneity. In this study, we proposed a latent feature-enhanced generative adversarial network model for the automatic extraction of individual brain tau deposition regions. The latent feature-enhanced generative adversarial network we propose can learn the distribution characteristics of tau PET images of cognitively normal individuals and output the abnormal distribution regions of patients. This model was trained and validated using 1131 tau PET images from multiple centres (with distinct races, i.e., Caucasian and Mongoloid) with different tau PET ligands. The overall quality of synthetic imaging was evaluated using structural similarity (SSIM), peak signal to noise ratio (PSNR), and mean square error (MSE). The model was compared to the fixed templates method for diagnosing and predicting AD. The reconstructed images archived good quality, with SSIM = 0.967 ± 0.008, PSNR = 31.377 ± 3.633, and MSE = 0.0011 ± 0.0007 in the independent test set. The model showed higher classification accuracy (AUC = 0.843, 95 % CI = 0.796-0.890) and stronger correlation with clinical scales (r = 0.508, P < 0.0001). The model also achieved superior predictive performance in the survival analysis of cognitive decline, with a higher hazard ratio: 3.662, P < 0.001. The LFGAN4Tau model presents a promising new approach for more accurate detection of individualized tau deposition. Its robustness across tracers and races makes it a potentially reliable diagnostic tool for AD in practice.

Enhancing generative adversarial network

The paper provides a comprehensive review of various GAN methods from the perspectives of theory, and applications. GAN algorithms' mathematical representations, and structures are detailed. The commonalities and differences among these GANs methods are compared. Theoretical issues related to GANs are explored, and typical applications in various fields are showcased. Future scope of research problems for GANs are also discussed in the paper.

Improving the spatial resolution of solar images using super-resolution diffusion generative adversarial networks

High-spatial-resolution solar images contribute to the study of small-scale structures on the Sun. The Helioseismic and Magnetic Imager (HMI) conducts continuous full-disk observations of the Sun at a fixed cadence, accumulating a wealth of observational data. However, the spatial resolution of HMI images is not sufficient to analyze the small-scale structures of solar activity. We present a new super-resolution (SR) method based on generative adversarial networks (GANs) and denoising diffusion probabilistic models (DDPMs) that can increase the spatial resolution of HMI images by a factor four. We propose a method called super-resolution diffusion GANs (SDGAN), which combines GANs and DDPMs for the SR reconstruction of HMI images. SDGAN progressively maps low-resolution (LR) images to high-resolution (HR) images through a conditional denoising process. It employs conditional GANs to simulate the denoising distribution and optimizes model results using nonsaturating adversarial loss and perceptual loss. This approach enables fast and high-quality reconstruction of solar images. We used high-spatial-resolution images from the Goode Solar Telescope (GST) as HR images and created a data set consisting of paired images from HMI and GST. We then used this data set to train SDGAN for the purpose of reconstructing HMI images with four times the original spatial resolution. The experimental results demonstrate that SDGAN can obtain high-quality HMI reconstructed images with just four denoising steps.

Stellar spectral template library construction based on generative adversarial networks

Stellar spectral template libraries play an important role in the automated analysis of stellar spectra. Synthetic template libraries cover a very large parameter space but suffer from poor matching with observed spectra. In this study, we propose a synthetic-to-observed spectral translation (SOST) method based on generative adversarial networks. The SOST method is able to calibrate synthetic spectra by converting them to the corresponding observed spectra. We applied this method to Kurucz synthetic spectra and observed spectra data from the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST). After that, we constructed a stellar spectral library with uniform and broad parameter distributions using the SOST-corrected Kurucz synthetic spectra. Our stellar spectral template library contains 2431 spectra spanning a parameter space of 3500 to 8000 K for effective temperature ($T_ eff $), 0.0 to 5.0 dex for surface gravity ($ g$), and -2.0 to 0.5 dex for metallicity Fe H $). The spectra in the library have a resolution of R sim 1800 and cover the wavelength range 3900-8700 In order to verify the accuracy of this template library, we used the template library and the template-matching algorithm to derive the parameters of the PASTEL database. Compared to measurements using the original synthetic template library, the accuracies of the three parameters eff g$, and Fe H $, are improved, from 140 K, 0.31 dex, and 0.21 dex to 121 K, 0.26 dex, and 0.13 dex, respectively. In addition, we re-parameterised more than six million stellar spectra released by LAMOST DR8.

Enhancing Skin Disease Classification: A Novel Approach With Tailored Loss Functions And SMOTE Sumeet Ghumare

Although skin disease like skin cancer is a common health problem worldwide, early identification is crucial to its recovery. By using medical photos, the rise of artificial intelligence (AI) has transformed the detection of skin diseases. Despite the reality that various deep learning models have been created with this goal in mind, class imbalance in multi class classification all poses a serious difficulty and imbalance among the classes. This work offers a novel strategy to reduce class imbalance in the categorization of skin diseases, leading to significant accuracy gains. One of the largest skin cancer datasets currently accessible, it consists of 33,106 dermoscopy photos from seven different kinds of skin diseases as a starting point. The main goal is to enhance skin disease classification algorithms’ performance on this imbalance dataset. We are going to use a two-pronged strategy to combat the class disparity and GAN (Generative adversarial network) Based Oversampling or Meta-learning in order to successfully equalize the representation of each class during training, we first design a balanced Dynamic Mini-Batch Size logic, SMOTE (Synthetic Minority Oversampling Technique) and Augmenter. Second, we provide a new loss function designed to take into account the unique features of the dataset. The outcomes are startling. These findings underscore the efficacy of our hybrid approach in training deep convolutional neural networks for imbalanced skin disease datasets. By combining data-level balancing techniques with a carefully designed loss function at the algorithm level, we address the challenge of among the classes, paving the way for more accurate and reliable skin cancer diagnoses. Key Words: Dermoscopy images, DenseNet169, Convolution neural network, SMOTE (Synthetic Minority Over-sampling Technique, GAN(Generative adversarial network), Mini-Batch Logic, Augmenter

A review of the applications of generative adversarial networks to structural and functional MRI based diagnostic classification of brain disorders

Structural and functional MRI (magnetic resonance imaging) based diagnostic classification using machine learning has long held promise, but there are many roadblocks to achieving their potential. While traditional machine learning models suffered from their inability to capture the complex non-linear mapping, deep learning models tend to overfit the model. This is because there is data scarcity and imbalanced classes in neuroimaging; it is expensive to acquire data from human subjects and even more so in clinical populations. Due to their ability to augment data by learning underlying distributions, generative adversarial networks (GAN) provide a potential solution to this problem. Here, we provide a methodological primer on GANs and review the applications of GANs to classification of mental health disorders from neuroimaging data such as functional MRI and showcase the progress made thus far. We also highlight gaps in methodology as well as interpretability that are yet to be addressed. This provides directions about how the field can move forward. We suggest that since there are a range of methodological choices available to users, it is critical for users to interact with method developers so that the latter can tailor their development according to the users' needs. The field can be enriched by such synthesis between method developers and users in neuroimaging.

Time Series Forecasting with Missing Data Using Generative Adversarial Networks and Bayesian Inference

This paper tackles the challenge of time series forecasting in the presence of missing data. Traditional methods often struggle with such data, which leads to inaccurate predictions. We propose a novel framework that combines the strengths of Generative Adversarial Networks (GANs) and Bayesian inference. The framework utilizes a Conditional GAN (C-GAN) to realistically impute missing values in the time series data. Subsequently, Bayesian inference is employed to quantify the uncertainty associated with the forecasts due to the missing data. This combined approach improves the robustness and reliability of forecasting compared to traditional methods. The effectiveness of our proposed method is evaluated on a real-world dataset of air pollution data from Mexico City. The results demonstrate the framework’s capability to handle missing data and achieve improved forecasting accuracy.

Generative adversarial network-based synthesis of contrast-enhanced MR images from precontrast images for predicting histological characteristics in breast cancer

Objective. Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a sensitive tool for assessing breast cancer by analyzing tumor blood flow, but it requires gadolinium-based contrast agents, which carry risks such as brain retention and astrocyte migration. Contrast-free MRI is thus preferable for patients with renal impairment or who are pregnant. This study aimed to investigate the feasibility of generating contrast-enhanced MR images from precontrast images and to evaluate the potential use of synthetic images in diagnosing breast cancer. Approach. This retrospective study included 322 women with invasive breast cancer who underwent preoperative DCE-MRI. A generative adversarial network (GAN) based postcontrast image synthesis (GANPIS) model with perceptual loss was proposed to generate contrast-enhanced MR images from precontrast images. The quality of the synthesized images was evaluated using the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The diagnostic performance of the generated images was assessed using a convolutional neural network to predict Ki-67, luminal A and histological grade with the area under the receiver operating characteristic curve (AUC). The patients were divided into training (n = 200), validation (n = 60), and testing sets (n = 62). Main results. Quantitative analysis revealed strong agreement between the generated and real postcontrast images in the test set, with PSNR and SSIM values of 36.210 ± 2.670 and 0.988 ± 0.006, respectively. The generated postcontrast images achieved AUCs of 0.918 ± 0.018, 0.842 ± 0.028 and 0.815 ± 0.019 for predicting the Ki-67 expression level, histological grade, and luminal A subtype, respectively. These results showed a significant improvement compared to the use of precontrast images alone, which achieved AUCs of 0.764 ± 0.031, 0.741 ± 0.035, and 0.797 ± 0.021, respectively. Significance. This study proposed a GAN-based MR image synthesis method for breast cancer that aims to generate postcontrast images from precontrast images, allowing the use of contrast-free images to simulate kinetic features for improved diagnosis.

Integrating Drone Imagery and AI for Improved Construction Site Management through Building Information Modeling

In the rapidly advancing field of construction, digital site management and Building Information Modeling (BIM) are pivotal. This study explores the integration of drone imagery into the digital construction site management process, aiming to create BIM models with enhanced object recognition capabilities. Initially, the research sought to achieve photorealistic rendering of point cloud models (PCMs) using blur/sharpen filters and generative adversarial network (GAN) models. However, these techniques did not fully meet the desired outcomes for photorealistic rendering. The research then shifted to investigating additional methods, such as fine-tuning object recognition algorithms with real-world datasets, to improve object recognition accuracy. The study’s findings present a nuanced understanding of the limitations and potential pathways for achieving photorealistic rendering in PCM, underscoring the complexity of the task and laying the groundwork for future innovations in this area. Although the study faced challenges in attaining the original goal of photorealistic rendering for object detection, it contributes valuable insights that may inform future research and technological development in digital construction site management.

Opinion mining from amazon reviews using dual interactive wasserstein namib beetle generative adversarial network

Opinion mining from amazon reviews using dual interactive wasserstein namib beetle generative adversarial network

Atomic Force Microscopy Simulations for CO-functionalized tips with Deep Learning

Abstract Atomic Force Microscopy (AFM) operating in the frequency modulation mode with a metal tip functionalized with a CO molecule \ph{is able to image} the internal structure of molecules with an unprecedented resolution. The interpretation of these images is often difficult, making the support of theoretical simulations important. Current simulation methods, particularly the most accurate ones, require expertise and resources to perform ab initio calculations for the necessary inputs (i.e charge density and electrostatic potential of the molecule). Here, we propose a computationally inexpensive and fast alternative to the physical simulation of these Atomic Force Microscopy (AFM) images based on a Conditional Generative Adversarial Network (CGAN), that avoids all force calculations, and uses as the only input a 2D ball–and–stick depiction of the molecule. We discuss the performance of the model when trained with differ- ent subsets extracted from the previously published QUAM-AFM database. Our CGAN reproduces accurately the intramolecular contrast observed in the simulated images for quasi–planar molecules, but has limitations for molecules with a substantial internal corrugation, due to the strictly 2D character of the input.

A novel approach for improving open scene text translation with modified GAN

AbstractText, as a vital tool for communication, is playing an imperative role in modern society. Precise high-level text translation systems are essential requirements in a wide range of real-world applications, such as robot navigation, industrial automation, image search, and instant translation. Regardless of improved research, a series of grand challenges may still become upon when translating text automatically in the real-world from open scene images. The difficulties mainly stem from multiplicity and inconsistency of text in open scenes, complication and obstruction of backgrounds, and deficient imaging conditions in uncontrolled circumstances for open scene images. The existing deep learning-based text translation systems do not eliminate the text for translation, and these applications just replace text on the reconstructed scene. To address the abovementioned shortcomings, this study proposed a novel approach for open scene text translation. Our system consists of five modules including scene text detection, text recognition, text elimination, text translation, and text insertion along with scene reconstruction. The novelty presented by our model lies in the idea of first eliminating the text from the open scene for accurate translation and then reconstructs the translated text on the image for its proper alignment. We specifically modified the existing generative adversarial network (GAN) architecture for improved performance of text elimination by introducing a novel strategy of text and scene concatenation to reduce the overall loss function. For this purpose, we created a synthetic dataset to train our GAN for text elimination module. Experiments on various standard text translation systems demonstrate that our integrated system is able to outperform state-of-the-art approaches in terms of result quality. We have achieved 90.87% of precision, 83.66% of recall, 87.116% of F1-score, and reduced both losses ($$l_1$$ l 1 and $$l_2$$ l 2 ) up to 50% which is remarkable upon state-of-the-art translation systems.

SCDC-Net: Detection and Classification of Skin Cancer using Deep q Neural Network with Hybrid Features

The increment in radiation in atmosphere causing wide spread of skin cancer disease around the globe. Cancer of the skin is one of the most lethal forms of the illness, and it is responsible for millions of deaths annually all over the world. Because of this, the early identification of skin cancer using computer-aided diagnostic (CAD) technology might potentially save a person's life.  However, skin cancer classification of performance is affected by hair artifacts and improper segmentation of lesion. The conventional methods were failed to achieve higher performance due to improper feature analysis. Therefore, to achieve the robust performance, this work was developed the transfer learning-based skin cancer detection and classification (SCDC-Net). Initially, UG-Net model was introduced to eliminate hair from skin lesions by enhancing the region of interest, which is developed by combining U-shaped deep learning network (U-Net) with generative adversarial networks (GAN). In addition, Hybrid U-Net (HU-Net) perform the segmentation operation for identifying the disease effected region of interest. Further, the features are extracted using gray level cooccurrence matrix (GLCM) matrix and discrete wavelet transform (DWT), which extracts the color, texture, statistical features. Finally, these combined characteristics are used to train a deep q neural network (DQNN) model, which then classifies a number of different skin cancer illness categories. The simulations conducted on real-time ISIC-2019 dataset proved that proposed SCDC-Net outperformed in all performance measures as compared to existing methods.

Application of deep learning model incorporating domain knowledge in international migration forecasting

PurposeForecasting population movement trends is crucial for implementing effective policies to regulate labor force growth and understand demographic changes. Combining migration theory expertise and neural network technology can bring a fresh perspective to international migration forecasting research.Design/methodology/approachThis study proposes a conditional generative adversarial neural network model incorporating the migration knowledge – conditional generative adversarial network (MK-CGAN). By using the migration knowledge to design the parameters, MK-CGAN can effectively address the limited data problem, thereby enhancing the accuracy of migration forecasts.FindingsThe model was tested by forecasting migration flows between different countries and had good generalizability and validity. The results are robust as the proposed solutions can achieve lesser mean absolute error, mean squared error, root mean square error, mean absolute percentage error and R2 values, reaching 0.9855 compared to long short-term memory (LSTM), gated recurrent unit, generative adversarial network (GAN) and the traditional gravity model.Originality/valueThis study is significant because it demonstrates a highly effective technique for predicting international migration using conditional GANs. By incorporating migration knowledge into our models, we can achieve prediction accuracy, gaining valuable insights into the differences between various model characteristics. We used SHapley Additive exPlanations to enhance our understanding of these differences and provide clear and concise explanations for our model predictions. The results demonstrated the theoretical significance and practical value of the MK-CGAN model in predicting international migration.

Wear monitoring of micro-milling tools based on Improved Siamese Neural Network

Micro-milling is employed for the precise machining of small components with intricate characteristics, and the quality of micro-milling is significantly influenced by the uncertainty surrounding tool wear. Monitoring the wear status of micro-milling tools holds great significance in enhancing machining efficiency and sustainability. In this work, wear monitoring of micro-milling tools method was proposed based on Improved Siamese Neural Network. The Improved Siamese Neural Network used two different global feature extraction networks. The features were input into the t improved Siamese neural network model for similarity score calculation and anomaly detection, which improves the detection accuracy of the abnormal state of micro-milling tool wear. In addition, Generative Adversarial Network (GAN) data augmentation is used to address the issue of abnormal data imbalance during the data preparation stage. The experiments show that the classification accuracy and the precision of the model on the micro-milling tool wear dataset is 92.15% and 93.32% respectively, indicating that the model has good stability and effectiveness.

Clinical feasibility of deep learning-based synthetic CT images from T2-weighted MR images for cervical cancer patients compared to MRCAT

This work aims to investigate the clinical feasibility of deep learning-based synthetic CT images for cervix cancer, comparing them to MR for calculating attenuation (MRCAT). Patient cohort with 50 pairs of T2-weighted MR and CT images from cervical cancer patients was split into 40 for training and 10 for testing phases. We conducted deformable image registration and Nyul intensity normalization for MR images to maximize the similarity between MR and CT images as a preprocessing step. The processed images were plugged into a deep learning model, generative adversarial network. To prove clinical feasibility, we assessed the accuracy of synthetic CT images in image similarity using structural similarity (SSIM) and mean-absolute-error (MAE) and dosimetry similarity using gamma passing rate (GPR). Dose calculation was performed on the true and synthetic CT images with a commercial Monte Carlo algorithm. Synthetic CT images generated by deep learning outperformed MRCAT images in image similarity by 1.5% in SSIM, and 18.5 HU in MAE. In dosimetry, the DL-based synthetic CT images achieved 98.71% and 96.39% in the GPR at 1% and 1 mm criterion with 10% and 60% cut-off values of the prescription dose, which were 0.9% and 5.1% greater GPRs over MRCAT images.

Synthesising microstructures of a partially frozen salty sand using voxel-based 3D generative adversarial networks

Generating synthetic material microstructures is essential in the numerical modelling of geomaterials. The occurrence of permafrost and saline groundwater overlapping regions is crucial in a series of phenomena, such as carbon emissions and subgrade settlements. The microstructure of geomaterials in these regions is of particular complexity because of the multiphase nature with salty water and ice crystals. This complexity renders existing generative models ineffective in synthesising their microstructures. Traditional generative methods are limited in the sense that require prior knowledge of material descriptors. Recently, machine learning generative models achieved unprecedented levels of performance and realism, but still lack the means to assess posterior error. This work aims to bridge the gap between traditional methods and deep learning generative models by assessing posterior image quality in the latter. A 3D Generative Adversarial Network (GAN) model is trained with image samples from an X-ray CT of a partially frozen salty sand. The metrics retained to assess posterior quality are particle fabric (shape parameter and anisotropy) and homogenised elastic coefficients obtained with Finite Element Method (FEM) simulations. A hyperparametric study on batch size and latent dimension serves to select the best configuration based on particle fabric. FEM simulations determine the deviation in the generated images elastic coefficients being 7.55% on average with respect to real samples. With 803 voxels, generated images are the largest up-to-date in a three-phase material, allowing to reach REV criteria. Applications range from the generation of microscales in double-scale models to the calibration of image processing tools.